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Abstract 

This paper discusses the results of a study comparing the psychometric qualities of 
two forms of an identical survey: one administered in paper-and-pencil format and the 
other administered in Web format. The survey addressed the topic of college course 
anxiety and was used to survey a sample of undergraduate students {N = 236). The 
psychometric qualities investigated include the overall reliability (internal consistency) 
and item analyses of the 56 individual survey items for both the paper-and-pencil and 
Web surveys. Results revealed a good deal of similarity between the two modes of 
survey delivery. Analysis of the paper-and-pencil survey data generated a Cronbach’s a 
equal to .88; analysis of the Web survey data generated a value of .89. Further informal 
comparisons of the item analyses revealed very similar patterns in the responses to 
individual items, indicating that the two modes of survey delivery were quite comparable. 
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A Comparison of the Psychometric Qualities of Surveys 
Administered by Web and Traditional Methods 

Background 

The Internet has had a substantial impact on the field of survey research (Shannon, 
Johnson, Searcy, & Lott, 2001). This is evidenced by the growing number of 
electronically administered surveys over the past several years. Web surveys — which 
Dillman (1998) has referred to as one type of “self-administered” survey — are an 
extremely promising method of data collection (Schillewaert, Langerak, & Duhamel, 
1998). Young and Ross (2000) state that the use of the Internet to collect data may be 
one of the most profound developments in survey research. Witte, Amoroso, and 
Howard (2000) indicated that Internet research is “an area marked by great potential but 
also little experience” (p. 180). Specifically, the World Wide Web can be used as a 
resource for obtaining and efficiently processing large amounts of data, often the type 
collected as a result of administering a survey (Carbonaro & Bainbridge, 2000). Unlike 
more traditional survey delivery methods (e.g., mail and telephone), it is not clear what 
principles should guide the construction and implementation of electronic surveys 
(Shannon et al., 2001). 

Advantages of Web surveys include a high rate of response, short time frame for the 
collection of responses, and time and cost savings. The Web certainly addresses the 
need for a less expensive and more expedient method of data collection (Heflich & Rice, 
1999; Schillewaert et al., 1998). Furthermore, several additional benefits of using the 
Web for data collection have been identified (Carbonaro & Bainbridge, 2000; Mertler, 
2001, 2002a, 2002b; Schillewaert et al., 1998; Shannon et al., 2001). These benefits 
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include a faster response, protection against the loss of data, easy transfer of data into a 
database for analysis, cost savings, convenience for the respondent, the possibility of 
wider geographic coverage, and a potentially better response rate — although this 
“advantage” is not uniformly agreed upon by the community of survey researchers 
(Matz, 1999). 

However, it is important to note, as with any method of data collection, there also 
exist disadvantages. These include the potentially nonrandom nature of the sample, 
unavailability of population lists, computer access to the survey, and various technology- 
related issues. Additional limitations include the inability to clearly define the population, 
lack of technological familiarity on the part of respondents or their willingness to use a 
computer to complete the survey, the potential for being able to identify respondents, 
and browser incompatibility problems (Carbonaro & Bainbridge, 2000; Schillewaert et al., 
1998; Shannon et al., 2001). 

One of the most substantial concerns about Web surveys is the potential nonrandom 
nature of the respondent group (Mertler, 2001 , 2002a, 2002b; Witte et al., 2000). 
However, the issue of nonrandomness is not unique to Web-based survey research and 
can be addressed through the maintenance of an accurate list of population members, 
when feasible. Survey research professionals have suggested that Web surveys be 
used primarily with specifically identifiable samples such as “in-house" employee groups 
(Shannon et al., 2001). Alternatively, Taylor (2000) has suggested that we remember 
that online data collection is not based on probability sampling, but rather on “volunteer" 
or “convenience" sampling. 

Beyond the strengths and weaknesses of this methodology, Carbonaro and 
Bainbridge (2000) have outlined several other issues with which researchers should be 
concerned. First, access to the survey must be as simple as possible for all respondents. 
The more complex the process of completing the survey, the lower the resultant 
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response rate will undoubtedly be. Second, the process must be designed such that 
respondents of the Web survey are able to complete it with the same relative ease as if 
they had received a traditional paper version. Third, some sort of security system is 
required, in order to maintain the integrity of the data, but also to put at ease the mind of 
the respondent. Finally, completion of the survey must require only minimal computer 
skills — including the ability to use an Internet browser, enter a specific URL, use a 
mouse, and type on a word processor. Matz (1999) and Shannon et al. (2001) also point 
out that a Web survey must somehow be publicized. Some mechanism must be used to 
direct potential respondents to the actual URL containing the survey, either by providing 
a link to the URL in an email message or by providing the URL in a cover letter. 

There exists somewhat of a void in the research literature on the topic of electronic 
surveys, in particular, comparing their effectiveness to that of mail surveys (Undenvood, 
Kim, & Metier, 2000). Often, the results of Web-based surveys differ when compared to 
written questionnaires and telephone surveys (Taylor, 2000), although Saphore (1999) 
found that there were no differences in the pattern of responses between a Web survey 
and an identical pencil-paper form of the same survey. Furthermore, he concluded that 
there were no differences in the psychometric qualities of the two forms of the survey. 

In another comparison-type study, Matz (1999) compared the responses received 
from a Web survey with those received from a traditional paper-and-pencil survey. She 
found no significant differences between the demographic characteristics — such as age 
and gender — of the respondents completing the Web survey and those completing a 
mailed, written survey. Additionally, she found no significant differences between the 
content or pattern of responses for either group. The only significant finding — albeit, a 
substantial one — was that the overall response rate for the paper survey (43%) was 
significantly higher than that for the Web survey group (33%). A finai result discussed by 
the researcher was that the paper instrument seemed to provide more flexibility to the 
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respondents. They could freely make comments about items they did not understand or 
felt were ambiguous. In spite of these last two findings, she concluded that Web surveys 
seemed to be a reasonable alternative — or, at least, equally suspect — to mail surveys. 

A final example of research comparing the psychometric characteristics of these two 
survey methodologies is the pilot study that preceded the study at hand. Mertler and 
Earley (2002) concluded that Web-based and traditional survey methodologies result in 
similar sets of psychometric qualities based on comparable obtained values for internal 
consistency reliability. Although not identical, the patterns of responses — as identified by 
contributions to the overall scale — across the two modes of delivery were also fairly 
comparable. 

Because of the technological skills required to develop and implement Web-based 
surveys, the driving force behind their use has not been survey professionals, but rather 
technology specialists (or at least someone with a background in technology) (Dlllman & 
Bowker, 2001; Shannon et al., 2001). In order to harness the potential for using the 
Internet for the collection of valid and reliable data, those most knowledgeable about 
survey research methodology — specifically with respect to causes and consequences of 
survey error (Dillman & Bowker, 2001) — must take an active role in its development as a 
viable methodology for data collection. Specifically, research is needed in order to 
compare responses from Web surveys and those conducted in more traditional manners 
(Dillman & Bowker, 2001). 

Pilot Study 

In Spring 2002, the researchers conducted a pilot study of the instrument and modes 
of delivery (Mertler & Earley, 2002). The purpose of this pilot study was twofold: (1 ) to 
compare the psychometric qualities of two versions (i.e., paper-and-pencil versus Web) 
of a survey. It was concluded that the two modes of delivery resulted in very similar 
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values for internal consistency reliability. Additionally, it was concluded that response 
patterns across individual items were fairly analogous. 

An important — albeit unintended — outcome of this pilot study dealt with the methods 
themselves. It was determined that soliciting participation for a Web-based survey 
should be done electronically, and not by providing the URL on a piece of paper, as was 
done during the pilot study. It was our belief that providing a URL on a piece of paper 
(e.g., in a cover letter) is nof equivalent to handing participants an actual paper-and- 
pencil survey. The latter is a more “direct” form of survey delivery — it is easier for the 
participant to respond. When participants are merely provided with the URL for a survey, 
it takes some initiative on their part to go to a computer, type in the correct URL, and 
then complete the survey — it creates somewhat of an “extra” step when compared to the 
more traditional mode of delivery. This methodological issue was addressed and 
appropriately modified in the present study. 

Purpose of the Study 

The primary purpose of this study was to add to the knowledge base regarding the 
use of Web-based surveys as a viable means of collecting data for educational research. 
Specifically, the researchers investigated and compared the relative effectiveness, 
psychometric qualities, and response patterns of two versions of the same survey: one 
delivered as a Web-based survey and the other as a paper survey. A secondary purpose 
was to assess the specific methods used in this study for data collection in order to 
refine the procedures for future extensions of this line of inquiry. 

Methods 

Participants 

In the Spring 2003 term, the researchers contacted the directors of the English 112 
introductory writing course and the Math 115 introductory statistics course to obtain 
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permission to deliver surveys to randomly selected sections. Directors and section 
instructors gave access to 15 writing sections and 6 statistics sections. 

The researchers then contacted course instructors to arrange times to visit their 
classes and introduce one of the two survey formats: paper-and-pencil or web-based. 
With scheduling and time constraints, researchers ended up visiting six English sections 
(n = 121 students) and five Math sections (n = 115 students). Three English and three 
Math sections (n = 117 students) received the paper survey in class. We asked students 
to return the survey via campus mail within two weeks. The other five sections (three 
English and two Math, n = 1 19 students) received only a letter introducing the study. 
They were told that they would be receiving an email message which would contain a 
link to the survey. After the class visits, we sent these students e-mail messages 
containing a link to the survey (thus alleviating the problem of having to type the URL 
themselves). Approximately two weeks later, we sent reminder e-mails to all students, 
regardless of the version they received. 

We offered an incentive in the form of a lottery to encourage higher response rates 
than we saw in our pilot study. We held two drawings for $100 prizes (one from the 
paper-based returns and one from the web-based returns) one week following the 
deadline for returns. For the paper-based survey, we included a separate slip of paper 
for respondents to write their names and telephone numbers; a third party then 
separated these slips from the surveys to maintain anonymity. For the web-based 
survey, we included a final web page after survey submission for students to provide 
their contact information; one researcher then received this information in an e-mail (the 
survey data went to another location). 
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Instrumentation 

In order to begin exploring the various statistics anxiety instruments in more depth, 
we combined items from two instruments measuring student attitudes toward statistics, 
both called the STATS (Rogness, 1993 and Sutarso, 1992), into one instrument which 
we called the Course Attitude Scale (see Figure 1 ). We removed duplicate items or 
items with similar wording to others, for 56 unique items in the final scale. We also re- 
worded all items on the new scale — worded to be generic to any course, not just 
statistics. So, an item such as “If I were not required to take a statistics course for my 
major, I wouldn’t take one” was changed to read, “If I were not required to take THIS 
COURSE for my major, I wouldn’t take it.” Pilot study data from Spring 2002 yielded a 
Cronbach’s a of 0.91 for the overall scale we created. We did not make any 
modifications to the survey itself after this pilot study. The paper-based version is a 
printed copy of the Web-based version, so students received identical surveys except for 
the mode of delivery. 



Insert Figure 1 about here 



Data Analyses 

For the two modes of survey delivery, analyses included the computation of internal 
consistency reliability (alpha) coefficients and individual item analyses. All analyses were 
conducted using SPSS (v. 11.0). 

Results 

Although the data resulting from the administration of the two surveys were analyzed 
for the entire group, the analyses were primarily based on “formal” (i.e., statistical) 
techniques followed by “informal” (i.e., nonstatistical) comparisons of the results by the 
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two subgroups — namely, the data resulting from the traditional paper-and-pencil method 
of delivery {n = 38) and those resulting from the Web form of the survey {n = 60). The 
return rate for the total survey was equal to 42%; the return rate for the paper-and-pencil 
administration was equal to 32%, while that for the Web administration was equal to 
50%. Compared to the results of the pilot study, the overall return rate improved by 14%. 
While the return rate remained nearly equal for the paper version of the survey (32% 
versus 34% in the pilot study), the rate for the Web survey more than doubled (50% 
versus 22% in the pilot study). 

Analysis of the overall scale comprised of 56 items for the entire group {N = 98) 
revealed an internal consistency measure (i.e., Cronbach’s index of internal consistency: 
also known as the alpha coefficient, or a) equal to .90, indicating a high degree of 
reliability. The internal consistency measure for the “paper” sample (a = .88) was nearly 
identical to that for the “Web” sample {a = .89). 

The resulting data were also analyzed at the level of the individual items. 

Specifically, the internal consistency of the overall scale with each individual item 
removed was determined. This is typically done in order to assess the contributions of 
individual items to the total scale. The resulting analysis reports an alpha (a) coefficient 
for the total scale minus the particular item. A “new” a coefficient (i.e., for the 55-item 
scale) that is tower than that for the 56-item scale indicates that the particular item did in 
fact contribute to the total scale; in other words, when the item was removed, the overall 
reliability decreased. In contrast, a “new” a coefficient that is higher \han that for the 
original scale indicates that the particular item did not contribute to the overall scale; in 
other words, when the item was removed, the overall reliability improved. 

Of the 56 items appearing on the survey administered in a traditional manner (i.e., 
paper-and-pencil), analyses of 13 items revealed improved internal consistency values 
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when they were individually removed from the scale. This represents an identical 
number of items when compared to the pilot results. The results of the individual item 
analyses for the survey administered via the Web were very similar. Of the 56 items, 14 
showed improved internal consistencies when removed from the scale. This was an 
improvement from 19 items identified during the pilot study. A summary of analyses of 
these individual items and the amounts of improvement in overall scale reliabilities are 
presented in Table 1 and Table 2, respectively. Furthermore, it is important to note that, 
upon informal comparison of the list of 13 items identified from the paper-and-pencil 
method and the 14 items identified from the Web delivery method, 7 items were common 
to both lists. 



Insert Table 1 about here 



Insert Table 2 about here 



Discussion 

Web-based surveys have begun to have a profound impact on survey research and 
are likely here to stay. Similar to previous studies — e.g., Underwood (2000), Matz 
(1999), Saphore (1999), and Mertler and Earley (2002) — this study has provided 
evidence of the similarities, at least with respect to psychometric qualities, between 
Web-based and paper-and-pencil survey methodologies. With respect to the primary 
purpose of this study, nearly identical values for internal consistency reliability were 
observed for the two versions of the survey. Although not identical, the patterns of 
responses — as identified by contributions to the overall scale — across the two modes of 
delivery were fairly comparable. With respect to the secondary purpose of this study, the 
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methodological modifications made after the initial pilot study resulted in 
improvements — at a minimum, with respect to the response rates. 

During the pilot study, we provided the URL for the Web-based version of the survey 
to potential respondents on paper as part of a cover letter. It later became apparent to us 
that soliciting participation for a Web-based survey should be done electronically, and 
not by providing the URL on a piece of paper. It was our belief that providing a URL on a 
piece of paper (e.g., in a cover letter) is not equivalent to handing participants an actual 
paper-and-pencil survey. The latter is a more “direct” form of survey delivery — it is easy 
for the participant to respond. When participants are directed to an electronic survey 
(e.g., on a Web page) in the manner used in the pilot, it takes some initiative on their 
part to go to a computer, type in the correct URL, and then complete the survey — it 
creates somewhat of an “extra” step when compared to the more traditional mode of 
delivery. Requesting the participation of individuals via email — as was done in the 
current study — allowed us to include in the “cover message” a link to the Web survey, 
thus making completion of the survey a much easier — and comparable — process. We 
believe that the more than double rate of response for the Web group of respondents 
serves as impressive evidence of this fact. 

As further evidence of this fact, even though a monetary incentive was offered to 
each group in an attempt to substantially increase the response rates when compared to 
those from the pilot study, only the Web group demonstrated such an increase. The Web 
group was offered a cash incentive and the response rate increased from 22% in the 
pilot to 50% here. The traditional group was offered the identical cash incentive and 
actually experienced a decrease — albeit a small one — in the rate of response. 

Research is beginning to demonstrate that a Web-based approach is a viable means 
of gathering survey data. However, further research must be conducted on this mode of 
delivery in order to pass judgment on its relative merits. There are several advantages to 
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electronic surveys, in general. These include such things as cost and time savings, as 
well as the ease of transfer of responses into a database. Are these advantages over 
paper-and-pencil surveys great enough to “counteract” the problems associated with 
nonrandom samples? Are potential respondents comfortable enough with technology to 
respond to surveys online? Or do they remain apprehensive about potentially realistic 
concerns such as anonymity, confidentiality, and security of their provided information? If 
the answer to the latter question is “yes,” do we as researchers end up with a 
“differential” type of random sample resulting from respondent self-selection? This, of 
course, is a concern for any type of survey research, but can we be sure that we are not 
ending up with a sample whose characteristics are different from those realized through 
paper-and-pencil forms of surveys? To address these and related issues, further 
research is most certainly called for. 

Web-based surveys must adhere to the principles of sound survey design and 
implementation. In some instances, those principles may need to be adapted in order to 
“match” the methodology, but without jeopardizing the integrity of the data or risking the 
anonymity and confidentiality of the respondents. As this methodology is refined, it will 
undoubtedly be used more frequently to conduct scholarly research. Empirical studies, 
such as this one, are necessary in order to guide this process of methodological 
refinement. 
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Figure 1 

The Course Attitude Scale 
(Web version) 
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Welcome... 



and thank you for taking a few minutes to complete the... 



Course Attitude Scale * 



DIRECTIONS: 

For each statement, please respond by indicating the extent to which each statement is true of you. Indicate your responses by clicking on the appropriate 

button. 

When you have finished, click on the *SUBMIT* button at the bottom of the page 
to send your responses. 





ENGLISH 112 


[math 115 


In which class did you hear about or receive this survey? 


1 1 


1 ^ 



Thinking only about the class you checked above, 
please indicate the extent to which each of the following statements is true of you... 



NUMBER 


STATEMENT 


NOT TRUE 
OF ME 










VERY TRUE 
OF ME 


1. 


Learning in this course is easy for me. 


1 


2 


3 


4 


5 


6 


2. 


I understand this course better than the majority of people in my class. 


1 


2 


3 


4 


5 


6 


3. 


This course makes me anxious. 


1 


2 


3 


4 


5 


6 


4. 


I like working with numbers. 


1 


2 


3 


4 


5 


6 


5. 


I enjoy working with calculators. 


1 


2 


3 


4 


5 


6 


6. 


I enjoy working with computers. 


1 


2 


3 


4 


5 


6 


7. 


My mother likes/liked this subject, so I will. 


1 


2 


3 


4 


5 


6 


8. 


My father likes/liked this subject, so I will. 


1 


2 


3 


4 


5 


6 


9. 


I feel that this course is only for men. 


1 


2 


3 


4 


5 


6 


10. 


I feel that this course is only for women. 


1 


2 


3 


4 


5 


6 


11. 


This course is very useful for my major. 


1 


2 


3 


4 


5 


6 


12. 


This course will improve my research ability. 


1 


2 


3 


4 


5 


6 


13. 


This course will be important for my career. 


1 


2 


3 


4 


5 


6 


14. 


I will be more competent in my subject area when I master this course. 


1 


2 


3 


4 


5 


6 


15. 


I can master this course with a great deal of effort. 


1 


2 


3 


4 


5 


6 


16. 


I study for this course regularly even when there is no specific assignment. 


1 


2 


3 


4 


5 


6 


17. 


I see my instructor when I do not understand something in this course. 


1 


2 


3 


4 


5 


6 


18. 


I ask questions in this course when I do not understand. 


1 


2 


3 


4 


5 


6 


19. 


I believe this course is a very interesting subject. 


1 


2 


3 


4 


5 


6 


20. 


If this course were not required for my major, I would not take it. 


1 


2 


3 


4 


5 


6 


21. 


I do not like this course. 


1 


2 


3 


4 


5 


6 


22. 


I am able to understand this subject as well as any other subject. 


1 


2 


3 


4 


5 


6 


23. 


I try to do as little work as 1 can in this course. 


1 


2 


3 


4 


5 


6 


24. 


1 doubt that any teacher could make me more comfortable with this subject. 


1 


2 


3 


4 


5 


6 


25. 


The right instuctor could probably make me enjoy this subject. 


1 


2 


3 


4 


5 


6 
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26. 


T feel insecure while taking exams. 


1 


2 


3 


4 


5 


6 


27. 


I am confident that I could do work which required me to use computers. 


1 


2 


3 


4 


5 


6 


28. 


No matter how hard I study, I will not do well in this course. 


1 


2 


3 


4 


5 


6 


29. 


I think being enrolled in this course is undesirable. 


1 


2 


3 


4 


5 


6 


30. 


1 would like to take more courses in this subject. 


1 


2 


3 


4 


5 


6 


31. 


No matter who teaches this course, 1 feel I would never be good at it. 


1 


2 


3 


4 


5 


6 


32. 


I think this course is worthwhile for most professionals in my field. 


1 


2 


3 


4 


5 


6 


33. 


I have trouble thinking mathematically. 


1 


2 


3 


4 


5 


6 


34. 


I have trouble thinking creatively. 


1 


2 


3 


4 


5 


6 


1 


I forget information that 1 actually know during an exam due to anxiety. 


1 


2 


3 


4 


5 


6 


36. 


I have little self-confidence when it comes to using computers. 


1 


2 


3 


4 


5 


6 


37. 


1 feel at ease doing the work for this course. 


1 


2 


3 


4 


5 


6 


38. 


Given a choice, I would prefer not to have to work with this material. 


1 


2 


3 


4 


5 


6 


39. 


I think that I am good at this material. 


1 


2 


3 


4 


5 


6 


40. 


I like working with words. 


1 


2 


3 


4 


5 


6 


41. 


I become very nervous prior to taking an exam. 


1 


2 


3 


4 


5 


6 


42. 


I try to avoid circumstances where I have to use this material. 


1 


2 


3 


4 


5 


6 


43. 


I would like to take another class in this subject even if I wasn’t required to do so. 


1 


2 


3 


4 


5 


6 


44. 


While taking a test, I feel calm. 


1 


2 


3 


4 


5 


6 


45. 


No matter who teaches this course, I would never like it. 


1 


2 


3 


4 


5 


6 


46. 


I think this course is an important part of my career education. 


1 


2 


3 


4 


5 


6 


47. 


The thought of working with a computer makes me nervous. 


1 


2 


3 


4 


5 


6 


48. 


I find it difficult to stop worrying about how I did on an exam when it is over. 


1 


2 


3 


4 


5 


6 


49. 


I think that using a computer is often more trouble than it’s worth. 


1 


2 


3 


4 


5 


6 


50. 


I wish that I didn’t have to take any course in this subject. 


1 


2 


3 


4 


5 


6 


51. 


I like the thought of using computers. 


1 


2 


3 


4 


5 


6 


52. 


I think studying this material is a beneficial experience. 


1 


2 


3 


4 


5 


6 


53. 


The right instructor could probably make this course easy for me. 


1 


2 


3 


4 


5 


6 


54. 


One of the parts of this course which I dread is having to use computers. 


1 


2 


3 


4 


5 


6 


55. 


I get anxiety if I even think about working on this course material. 


1 


2 


3 


4 


5 


6 


56. 


I would never consider taking this course as an elective. 


1 


2 


3 


4 


5 
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I Submit Survey! 



Items on this survey were adapted with permission from two existing instruments: "Student Attitude Toward Statistics" (Rogness, 1993) and "Students' 

Attitudes Toward Statistics" (Sutarso, 1992), 



Thank you for participating in this research study. 

If you have questions or comments about this survey, please feel free to contact us... 



Mark A. Earley, Ph.D. 
Bowling Green State University 
Bowling Green, Ohio 43403 
earlevm(d)b£net. besu. edu 



Craig A. Mertler, Ph.D. 
Bowling Green State University 
Bowling Green, Ohio 43403 
mertler(a)bsnet. besu. edu 
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Table 1 

Identified Items and Recalculated Reliabilities for Item Analysis of Data Resulting from 
Paper-and-pencil Survey 



Survey Item Number 


Alpha Coefficient 
(If Item Deleted) ® 


Amount of Change 
In Reliability 


Item 3 


.8838 


+.0079 


Item 8 ^ 


.8763 


+.0004 


Item 16^ 


.8769 


+.0010 


Item 18 


.8788 


+.0029 


Item 24 


.8801 


+.0042 


Item 25 


.8777 


+.0018 


Item 27 


.8784 


+.0025 


Item 35 


.8760 


+.0001 


Item 36 


.8778 


+.0019 


Item 41 ^ 


.8762 


+.0003 


Item 47 


.8765 


+.0006 


Item 48 ^ 


.8807 


+.0048 


Item 53 


.8773 


+.0014 



® Alpha coefficient of the total scale was equal to .8759 (rounded to .88). 

^ These items were also identified following the item analysis of data resulting from the 
Web-administered survey. 
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Table 2 

Identified Items and Recalculated Reliabilities for Item Analysis of Data Resulting from 
Web Survey 



Survey Item Number 


Alpha Coefficient 
(If Item Deleted) ® 


Amount of Change 
In Reliability 


Item 3 ^ 


.8945 


+.0003 


Item 4 


.8958 


+.0016 


Item 5 


.8963 


+.0021 


Item 8 ^ 


.8944 


+.0002 


Item 10 


.8944 


+.0002 


Item 16“" 


.8953 


+.0011 


Item 23 


.8943 


+.0001 


Item 24 


.8947 


+.0005 


Item 25 


.8959 


+.0017 


Item 33 


.8957 


+.0015 


Item 41 


.8946 


+.0004 


Item 48 


.8968 


+.0026 


Item 49 


.8952 


+.0010 


Item 54 


.8947 


+.0005 



Alpha coefficient of the total scale was equal to .8942 (rounded to .89). 

These items were also identified following the item analysis of data resulting from the 
paper-and-pencil-administered survey. 
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